Speech and action sequences are both continuous information streams that must be successfully segmented into constituent sub-units in order to be understood…
|
Unsuccessful speech segmentation. |
Successful action segmentation. |
In both the speech and action domain, we know this segmentation task is achieved via a combination of top-down and bottom-up processing.
Top-down processing involves the application of pre-existing knowledge to determine where boundaries between phrases occur. Bottom-up processing involved properties of the stimulus to determine boundary location.
Work with adults has highlighted top-down and bottom-up cues that support segmentation of speech and action. For example:
|
|
|
|
|
|
Listeners apply their knoweldge of word meanings and grammar to determine the locations of boundaries in speech (e.g. Mattys et al., 2007). |
Prosodic cues (e.g. pasue and pre-boundary lengthening) are produced at phrase boundaries (e.g. Wagner & Watson, 2010), and listeners detect these cues to determine the location of phrase boundaries in speech (e.g. Schafer et al., 2000). |
|
|
Observers track the goals and intentions of an actor, and map boundaries to moments of goal-achievement (e.g. Levine et al., 2017). |
The movement itself contains kinematic cues to boundaries between actions. These kinematic cues include pause and pre-boundary lengthening (Hilton et al., 2019), and observers make use of these kinematic cues to determine the location of boundaries in the action sequence (Hemeren & Thill, 2010). |
Infants’ access to top-down processes is restricted, because they do not yet possess the knowledge/experience.
In speech, it has been proposed that infants therefore initially capitalise on bottom-up cues (prosody) to segment the stream (Prosodic Bootstrapping Account; Gleitman & Wanner, 1982 )
However, little is known about infants’ processing of bottom-up cues during action segmentation.
In parallel to early speech processing, infants may capitalize on kinematic boundary cues to initially segment actions, especially when the actions are unfamiliar or not goal-directed.
|
|
|
|
These characters were then animated to perform sequences of three actions.
Two action sequences were defined:
On no-boundary trials each sequence was shown as a single continuous sequence.
On boundary trials, a boundary was signalled between the second and final action.
On boundary trials, the boundary was signalled by two kinematic boundary cues:
| Element | Duration (ms) | Duration (ms) |
|---|---|---|
| still frame | 1000 | 1000 |
| action 1 | 600 | 600 |
| action 2 | 600 | 840 |
| pause | 0 | 350 |
| action 3 | 750 | 750 |
Each character performed both sequences with and without a boundary, resulting in a final stimuli set of 12 videos (3 characters x 2 sequences x 2 trial types).
Below, you can see an example of a no-boundary trial and a boundary trial.
12-month-old infants (N = 23) from German-speaking households contributed data.
| Condition | Mean no. of trials | Range |
|---|---|---|
| no-boundary | 23.2 | 15 - 34 |
| boundary | 25.0 | 14 - 37 |
Infants were shown the 12 stimulus videos in a ranomized order until the infants became bored and thus looked consistently away from the screen. Including breaks and pauses, we typically were able to record EEG from the infants for ~ 10 minutes.
EEG was recorded from 30 electrodes.
9 of these electrodes served as critical electrodes for analysis (F3, Fz, F4, C3, Cz, C4, P3, Pz, P4).
Were infants sensitive to the kinematic boundary cues?
To answer this question, we examined the ERP from the onset of the pre-boundary lengthening (i.e. the mid-point of the second action) until the offset of the final action.
To examine whether infants processed the kinematic boundary cues, we examined the ERP in this time interval for evidence of the Closure Positive Shift.
Closure Positive Shift (CPS):
An ERP component initially discovered in response to prosodic boundary cues in speech (Steinhauer et al., 1999) . This component is a slow, broadly distributed positivity in the ERP that begins around the onset of the boundary and lasts approximately 500 ms (Boegels et al., 2011) .
For every trial, segments between the mid-point of the second action and the mid-point of the third action were exported for analysis. The maximum amplitude in each segment was calculated, resulting in an analysis of mean maximum amplitude during this time interval.
Repeated measures ANOVA (condition: no-boundary vs. boundary x region: frontal vs. central vs. posterior):
| effect | F | df | p | \(\eta_{G}^{2}\) |
|---|---|---|---|---|
| condition | 73.11 | 1, 26 | <.001 | 0.342 |
| region | 19.08 | 2, 52 | <.001 | 0.065 |
| condition*location | 2.70 | 2, 52 | 0.077 | 0.004 |
Do kinematic boundary cues modulate processing of subsequent actions?
Negative central (Nc) component:
A negative peak in the ERP over fronto-central electrodes emerging between 300 and 900 ms following stimulus onset, implicated in attentional processing (e.g., Nelson & Collins, 1991; Reynolds & Richards, 2005) . Has recently been taken as a measure of action processing during infancy, reflecting attention to and encoding of an individual actions (Monroy et al., 2019).
For every trial, the Nc was analysed by exporting the minimum amplitude from the ERP in the 250 ms - 750 ms time interval following the onset of each action. The mean minimum amplitude was then averaged across six fronto-central electrodes (F3, Fz, F4, C3, Cz, C4), and analysed with a 3 (action: first, second, final) x 2 (condition: boundary, no-boundary) repeated measures ANOVA.
| effect | F | df | p | \(\eta_{G}^{2}\) |
|---|---|---|---|---|
| condition | 0.65 | 1, 26 | 0.426 | 0.003 |
| action | 4.16 | 2, 52 | 0.021 | 0.021 |
| condition*action | 7.73 | 2, 52 | 0.001 | 0.026 |
| We are interested in how infants process boundaries between individual actions of an action sequence. |
| Work with adults suggests that kinematic cues (properties of the movement) can signal the location of boundaries in action sequences. |
| We presented 12-month-old infants with cartoon action sequences while recording EEG. |
| Half of the sequences contained kinematic boundary cues (pre-boundary lengthening and pause). |
| We found evidence of an ERP component indicating the infants detected and processes the kinematic boundary cues. |
| The kinematic cues also modulated infants’ processing of subsequent actions. |
| We contend that these low-level kinematic cues play a role in early action segmentation and processing. |
Twitter Icon on pdf from Freepik from www.flaticon.com